An Exploratory Study of Masked Face Recognition with Machine Learning Algorithms

Automated face recognition is a widely adopted machine learning technology for contactless identification of people in various processes such as automated border control, secure login to electronic devices, community surveillance, tracking school attendance, workplace clock in and clock out. Using face masks have become crucial in our daily life with the recent world-wide COVID-19 pandemic. The use of face masks causes the performance of conventional face recognition technologies to degrade considerably. The effect of mask-wearing in face recognition is yet an understudied issue. In this paper, we address this issue by evaluating the performance of a number of face recognition models which are tested by identifying masked and unmasked face images. We use six conventional machine learning algorithms, which are SVC, KNN, LDA, DT, LR and NB, to find out the ones which perform best, besides the ones which poorly perform, in the presence of masked face images. Local Binary Pattern (LBP) is utilized as the feature extraction operator. We generated and used synthesized masked face images. We prepared unmasked, masked, and half-masked training datasets and evaluated the face recognition performance against both masked and unmasked images to present a broad view of this crucial problem. We believe that our study is unique in elaborating the mask-aware facial recognition with almost all possible scenarios including half_masked-to-masked and half_masked-to-unmasked besides evaluating a larger number of conventional machine learning algorithms compared the other studies in the literature. © 2023 IEEE.

Masked Face Recognition System using Extended VGG-19

The scientific community has observed several issues as a result of COVID-19, both directly and indirectly. The use of face mask for health protection is crucial in the current COVID-19 scenario. Besides, ensuring the security of all people, from individuals to the state system, financial resources, diverse establishments, government, and non-government entities, is an essential component of contemporary life. Face recognition system is one of the most widely used security technology in modern life. In the presence of face masks, the performance of the current face recognition systems is not satisfactory. In this paper, we investigate a flexible solution that could be employed to recognize masked faces effectively. To do this, we develop a unique dataset to recognize the masked face, consisting of a frontal and lateral face with a mask. We propose an extended VGG19 deep model to improve the accuracy of the masked face recognition system. Then, we compare the accuracy of the proposed framework to that of well-known deep learning techniques, such as the standard Convolutional Neural Network (CNN) and the original VGG19. The experimental results demonstrate that the proposed extended VGG19 outperforms the investigated approaches. Quantitatively, the proposed model recognizes the frontal face with the mask with high accuracy of 96%. © 2022 IEEE.

Masked Face Recognition Using Multi-Task Cascaded Convolutional Networks and FaceNet

Face masks have been widely used since the start of the COVID-19 pandemic. Facial detection and recognition technologies, such as the iPhone's Face ID, heavily rely on seeing the facial features that are now obscured due to wearing a face mask. Currently, the only way to utilize Face ID with a mask on is by having an Apple Watch as well. As such, this paper intends to find initial means of a reliable personal facial recognition system while the user is wearing a face mask without having the need for an Apple Watch. This may also be applicable to other security systems or measures. Through the use of Multi-Task Cascaded Convolutional Networks or MTCNN, a type of neural network which identifies faces and facial landmarks, and FaceNet, a deep neural network utilized for deriving features from a picture of a face, the masked face of the user could be identified and more importantly be recognized. Utilizing MTCNN, detecting the masked faces and automatically cropping them from the raw images are done. The learning phase then takes place wherein the exposed facial features are given emphasis while the masks themselves are excluded as a factor in recognition. Data in the form of images are acquired from taking multiple pictures of a certain individual's face as well as from repositories online for other people's faces. Images used are taken in various settings or modes such as different lighting levels, facial angles, head angles, colors and designs of face masks, and the presence or absence of glasses. The goal is to recognize whether it is the certain individual or not in the image. The training accuracy is 99.966% while the test accuracy is 99.921%. © 2022 IEEE.

Ensemble Learning using Transformers and Convolutional Networks for Masked Face Recognition

Wearing a face mask is one of the adjustments we had to follow to reduce the spread of the coronavirus. Having our faces covered by masks constantly has driven the need to understand and investigate how this behavior affects the recognition capability of face recognition systems. Current face recognition systems have extremely high accuracy when dealing with unconstrained general face recognition cases but do not generalize well with occluded masked faces. In this work, we propose a system for masked face recognition. The proposed system comprises two Convolutional Neural Network (CNN) models and two Transformer models. The CNN models have been fine-tuned on FaceNet pre-trained model. We ensemble the predictions of the four models using the majority voting technique to identify the person with the mask. The proposed system has been evaluated on a synthetically masked LFW dataset created in this work. The best accuracy is obtained using the ensembled models with an accuracy of 92%. This recognition rate outperformed the accuracy of other models and it shows the correctness and robustness of the proposed model for recognizing masked faces. The code and data are available at https://github.com/Hamzah-Luqman/MFR. © 2022 IEEE.

IoT-enabled Contactless Doorbell with Facial Recognition

The COVID-19 epidemic has altered lifestyles all across the globe, causing people to take additional safety precautions and make using a face mask a requirement. Face masks are becoming more popular, making it occasionally challenging for people to recognize other people. Children and the elderly in particular would have trouble identifying their masked guests, which poses a serious hazard because thieves or burglars would take advantage of the situation. In this study, a system was created using IoT and deep learning technologies that works as a unit to offer a contactless solution to the ongoing COVID-19 pandemic while also enabling home owners to keep track of their visitors and receive notifications when someone comes over. The contactless doorbell was created with the help of a Raspberry Pi and a modified ResNet-50 model using ArcFace loss as the feature extractor to efficiently extract visible features from a masked face and support very accurate recognition. Due to the lack of a real masked face dataset with sufficient data, this study used a data augmentation method to add masks to face images from a dataset. The model was able to achieve a recognition accuracy of 98.27% when evaluated using a masked LFW dataset. Furthermore, testing the face recognition model in real-Time with limited users, each with and without a mask yielded an accuracy of 100% in unmasked facial recognition and 90% on masked facial recognition. © 2023 IEEE.

Improving Masked Face Recognition Using Dense Residual Unit Aided with Quadruplet Loss

All over the world, people are wearing face masks and practising social distancing to protect themselves against the Coronavirus disease (COVID-19). The need for contactless biometric systems has increased to avoid the common point of contact. Among contactless biometric systems, facial recognition systems are the most economical and effective ones. Conventional face recognition systems rely heavily upon the facial features of the eyes, nose, and mouth. But due to people wearing face masks, the important facial features of the nose and mouth get hidden under the mask, resulting in degraded performance by the facial recognition systems on masked faces. In this paper, we propose a Dense Residual Unit (DRU) aided with Quadruplet loss on top of existing facial recognition systems. This solution tries to unveil the masked faces by producing embeddings for masked faces, which are similar to embeddings of unmasked faces of the same identity but different from embeddings of different identities. We have evaluated our method using two pre-trained facial recognition models' backbones, i.e. Resnet-101 and MobileFaceNet, and upon two datasets, among them, one is a real-world dataset, i.e. MFR2, and one is a simulated masked face dataset i.e. masked version of LFW. We have achieved improvement in the performance of masked face recognition in terms of False Match Rate, False Non-Match Rate, Fisher Discriminant ratio, and Equal Error Rate. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Toward High-quality Face-Mask Occluded Restoration

Face-mask occluded restoration aims at restoring the masked region of a human face, which has attracted increasing attention in the context of the COVID-19 pandemic. One major challenge of this task is the large visual variance of masks in the real world. To solve it we first construct a large-scale Face-mask Occluded Restoration (FMOR) dataset, which contains 5,500 unmasked images and 5,500 face-mask occluded images with various illuminations, and involves 1,100 subjects of different races, face orientations, and mask types. Moreover, we propose a Face-Mask Occluded Detection and Restoration (FMODR) framework, which can detect face-mask regions with large visual variations and restore them to realistic human faces. In particular, our FMODR contains a self-adaptive contextual attention module specifically designed for this task, which is able to exploit the contextual information and correlations of adjacent pixels for achieving high realism of the restored faces, which are however often neglected in existing contextual attention models. Our framework achieves state-of-the-art results of face restoration on three datasets, including CelebA, AR, and our FMOR datasets. Moreover, experimental results on AR and FMOR datasets demonstrate that our framework can significantly improve masked face recognition and verification performance. © 2023 Association for Computing Machinery.

An efficient masked-face recognition technique to combat with COVID-19

Since the major COVID-19 pandemic hit the world, using face masks has become an important part of our daily lives. Not wearing masks for a moment when out¬side can be a life-threatening mistake. But the mask covers the lower portion of the face region. So we were not able to extract important features from the lower region. That is why in this paper, an efficient face recognition scheme has been presented by integrating discrete curvelet transform (DCT), compressive sensing (CS), and principal component analysis (PCA) to improve the face recognition rate of masked faces. The use of discrete curvelet transform (DCT) provides an enhancement of edge information of face images by applying an image fusion technique. To extract the feature vector in lower dimensional feature space, PCA has been applied on the fused images. Finally, the performance of this proposed technique is tested by using compressive sensing-based classifier. Extensive exper¬iments are simulated on masked and unmasked faces, and our method performs better than the conventional PCA. © 2023 Scrivener Publishing LLC. All rights reserved.

Masked Face Recognition Dataset and Application

During COVID-19 coronavirus epidemic, almost everyone wears a mask to prevent the spread of virus. It raises a problem that the traditional face recognition model basically fails in the scene of face-based identity verification, such as security check, community visit check-in, etc. Therefore, it is imminent to boost the performance of masked face recognition. Most recent advanced face recognition methods are based on deep learning, which heavily depends on a large number of training samples. However, there are presently no publicly available masked face recognition datasets, especially real ones. To this end, this work proposes three types of masked face datasets, including Masked Face Detection Dataset (MFDD), Real-world Masked Face Recognition Dataset (RMFRD) and Synthetic Masked Face Recognition Dataset (SMFRD). Besides, we conduct benchmark experiments on these three datasets for reference. As far as we know, we are the first to publicly release large-scale masked face recognition datasets that can be downloaded for free at https://github.com/X-zhangyang/Real-World-Masked-Face-Dataset.. IEEE

Joint Masked Face Recognition and Temperature Measurement System Using Convolutional Neural Networks.

With the outbreak of COVID-19, epidemic prevention has become a way to prevent the spread of epidemics. Many public places, such as hospitals, schools, and office places, require disinfection and temperature measurement. To implement epidemic prevention systems and reduce the risk of infection, it is a recent trend to measure body temperature through non-contact sensing systems with thermal imaging cameras. Compared to fingerprints and irises, face recognition is accurate and does not require close contact, which significantly reduces the risk of infection. However, masks block most facial features, resulting in the low accuracy of face recognition systems. This work combines masked face recognition with a thermal imaging camera for use as an automated attendance system. It can record body temperature and recognize the person at the same time. Through the designed UI system, we can search the attendance information of each person. We not only provide the design method based on convolutional neural networks (CNNs), but also provide the complete embedded system as a real demonstration and achieve a 94.1% accuracy rate of masked face recognition in the real world. With the face recognition system combined with a thermal imaging camera, the purpose of screening body temperature when checking in at work can be achieved.

Masked face recognition with convolutional visual self-attention network.

With the global outbreak of COVID-19, wearing face masks has been actively introduced as an effective public measure to reduce the risk of virus infection. This measure leads to the failure of face recognition in many cases. Therefore, it is very necessary to improve the recognition performance of masked face recognition (MFR). Inspired by the successful application of self-attention in computer vision, we propose a Convolutional Visual Self-Attention Network (CVSAN), which uses self-attention to augment the convolution operator. Specifically, this is achieved by connecting a convolutional feature map, which enforces local features, to a self-attention feature map that is capable of modeling long-range dependencies. Since there is currently no publicly available large-scale masked face data, we generate a Masked VGGFace2 dataset based on the face detection algorithm to train the CVSAN model. Experiments show that the CVSAN algorithm significantly improves the performance of MFR compared to other algorithms.

An approach to unveil the masked faces: Occluded Face-Recognition System

Face recognition is one of the most widely used biometric identification systems due to its practicality and ease of use. The COVID-19 outbreak has recently expanded rapidly over the world, posing a major threat to people's health and economic well-being. Using masks in public places is an efficient approach to prevent the spread of infections. However, due to the absence of facial features detail, a masked face recognition system is a difficult task. We offer a technique to identify the masked faces in this project.Masked face recognition is a subset of occluded face recognition that requires prior knowledge of the obscured portion of the targeted face. Occluded face recognition is a current research topic that has captured the interest of the computer vision community. Occluded face recognition systems have previously been focused on detecting and recognizing an individual's face in the wild when the occluded part of the face is in a random form and position. Meanwhile, the nose, mouth, and cheeks of a masked face are frequently hidden. The eyes, brows, and forehead may be the only remaining clear areas. As a result, a masked face recognition system could effectively concentrate on analyzing traits that can be derived from the subject's uncovered portions, such as the eyes, brows, and forehead. © 2022 IEEE.

Toward masked face recognition: An effective facial feature extraction and refinement model in multiple scenes

With the impact of the COVID‐19 epidemic, the demand for masked face recognition technology has increased. In the process of masked face recognition, some problems such as less feature information and poor robustness to the environment are obvious. The current masked face recognition model is not quantified enough for feature extraction, there are large errors for faces with high similarity, and the categories cannot be clustered during the detection process, resulting in poor classification of masks, which cannot be well adapted to changes in multiple environments. To solve current problems, this paper designs a new masked face recognition model, taking improved Single Shot Multibox Detector (SSD) model as a face detector, and replaces the input layer VGG16 of SSD with Deep Residual Network (ResNet) to increase the receptive field. In order to better adapt to the network, we adjust the convolution kernel size of ResNet. In addition, we fine‐tune the Xception network by designing a new fully connected layer, and reduce the training cycle. The weights of the three input samples including anchor, positive and negative are shared and clustered together with triplet network to improve recognition accuracy. Meanwhile, this paper adjusts alpha parameter in triplet loss. A higher value of alpha can improve the accuracy of model recognition. We further adopt a small trick to classify and predict face feature vectors using multi‐layer perceptron (MLP), and a total of 60 neural nodes are set in the three neural layers of MLP to get higher classification accuracy. Moreover, three datasets of MFDD, RMFRD and SMFRD are fused to obtain high‐quality images in different scenes, and we also add data augmentation and face alignment methods for processing, effectively reducing the interference of the external environment in the process of model recognition. According to the experimental results, the accuracy of masked face recognition reaches 98.3%, it achieves better results compared with other mainstream models. In addition, the hyper‐parameters tuning experiment is carried out to improve the utilization of computing resources, which shows better results than the indicators of different networks.

Masked Face Recognition Using Histogram-Based Recurrent Neural Network.

Masked face recognition (MFR) is an interesting topic in which researchers have tried to find a better solution to improve and enhance performance. Recently, COVID-19 caused most of the recognition system fails to recognize facial images since the current face recognition cannot accurately capture or detect masked face images. This paper introduces the proposed method known as histogram-based recurrent neural network (HRNN) MFR to solve the undetected masked face problem. The proposed method includes the feature descriptor of histograms of oriented gradients (HOG) as the feature extraction process and recurrent neural network (RNN) as the deep learning process. We have proven that the combination of both approaches works well and achieves a high true acceptance rate (TAR) of 99 percent. In addition, the proposed method is designed to overcome the underfitting problem and reduce computational burdens with large-scale dataset training. The experiments were conducted on two benchmark datasets which are RMFD (Real-World Masked Face Dataset) and Labeled Face in the Wild Simulated Masked Face Dataset (LFW-SMFD) to vindicate the viability of the proposed HRNN method.

MmLwThV framework: A masked face periocular recognition system using thermo-visible fusion.

In wake of COVID-19, the world has adapted to a new order. People have started wearing mask on their faces to prevent getting infected. The present face recognition models are no longer proving to be efficient in the current circumstances. This is because, most of the informative part of the face is covered by mask. The periocular recognition therefore holds the key to future of face recognition. However, the periocular region proves to be insufficiently enough to generate highly discriminative features. Also, most of the pre-COVID-19 algorithms fail to work in cases, where the number of training images available is very less. We propose a lightweight periocular recognition framework that uses thermo-visible features and ensemble subspace network classifier to improve upon the existing periocular recognition systems named as Masked Mobile Lightweight Thermo-visible Face Recognition (MmLwThV). The framework successfully improves the accuracy over a single visible modality by mitigating the effect of noise present in the thermo-visible features. The experiments on WHU-IIP dataset and an in-house collected dataset named, CVBL masked dataset, successfully validate the efficacy of our proposed framework. The MmLwFR framework is lightweight and can be easily deployed on mobile phones with a visible and an infrared camera.

MASKED FACE RECOGNITION VIA SELF-ATTENTION BASED LOCAL CONSISTENCY REGULARIZATION

With the COVID-19 pandemic, one critical measure against infection is wearing masks. This measure poses a huge challenge to the existing face recognition systems by introducing heavy occlusions. In this paper, we propose an effective masked face recognition system. To alleviate the challenge of mask occlusion, we first exploit RetinaFace to achieve robust masked face detection and alignment. Secondly, we propose a deep CNN network for masked face recognition trained by minimizing ArcFace loss together with a local consistency regularization (LCR) loss. This facilitates the network to simultaneously learn globally discriminative face representations of different identities together with locally consistent representations between the non-occluded faces and their counterparts wearing synthesized facial masks. The experiments on the masked LFW dataset demonstrate that the proposed system can produce superior masked face recognition performance over multiple state-of-the-art methods. The proposed method is implemented in a portable Jetson Nano device which can achieve real-time masked face recognition. © 2022 IEEE.

Fast R-CNN based Masked Face Recognition for Access Control System

Masked face recognition is becoming an essential requirement in most of the facial recognition based access control and authentication systems, particularly after the Covid-19 pandemic. The work analyses the capability of Region Based Convolutional Neural Networks (R-CNN) for masked face detection and demonstrates a facial recognition system with R-CNN in hardware. R-CNN uses Region Proposal Networks (RPN) that can extract non-occluded region on an image and feed it to a Deep Neural Network for recognition. The R-CNN classifier running on the region containing the non-occluded part of the face will be used for classification in case of a masked face. In case of recognition of unmasked face, the classifier will be run on the region containing the face. By this way, the system will be able to recognize face for both cases. Python modules like opencv, numpy have been used for image pre-processing, while Tensorflow has been used for image classification. Custom dataset is used for training. The trained deep learning model is evaluated using a confusion matrix heat-map which can be used to know the reliability of the model. The demonstration system consists of a Raspberry Pi module connected to a door actuator and a camera. On successful authentication, the system opens the door. © 2022 IEEE.

Recognition of Faces Wearing Masks Using Skip Connection Based Dense Units Augmented With Self Restrained Triplet Loss

Facial recognition-based systems are the most efficient and cost-effective of all the contactless biometric verification systems available. But, in the COVID-19 scenario, the performance of available facial recognition systems has been affected badly due to the presence of masks on people's faces. Various studies have reported the degradation of the performance of facial recognition systems due to masks. Therefore, there is a need for improvement in the performance of currently available facial recognition algorithms. In this research, we propose using Skip Connection based Dense Unit (SCDU) trained with Self Restrained Triplet Loss, to handle the embeddings produced by existing facial recognition algorithms for masked images. The SCDU is trained to make facial embeddings for unmasked and masked images of the same identity similar, as well as, embeddings for unmasked and masked images of different identities dissimilar. We have evaluated our results on the LFW dataset with synthetic masks as well as the real-world masked face recognition dataset, i.e., MFR2 and achieved improvement in verification performance in terms of Equal Error Rate, False Match Rate, False Non-Match Rate, and Fisher discriminant ratio. © 2022 IEEE.

Masked Face Recognition Using FaceNet

The global epidemic of COVID-19 has seriously affected people's life. To prevent and control the outbreak, people are required to wear masks, which poses a formidable challenge to the existing face recognition system. A masked face recognition method based on FaceNet is proposed to tackle the problems. In this paper, a smaller model based on the Inception-ResN et Vl model is proposed. The main idea is to reduce filter numbers in each inception block while maintaining the whole structure. The reduced version has much fewer parameters to compute and can recognize faces with and without masks. Comprehensive experiments on both masked and unmasked datasets have been conducted. With 99.79% test accuracy in the masked MS-Celeb-1M dataset, the model trained in this paper can be integrated into existing face recognition programs designed to recognize faces for verification purposes. © 2022 IEEE.

Masked Face Recognition Using MobileNetV2

Masked face recognition has made great progress in the field of computer vision since the popularity of COVID-19 epidemic in 2020. In countries with severe outbreaks, people are required to wear masks in public. The current face recognition methods, which take use of the whole face as input data, are quite well established. However, while people are use of face masks, it will reduce the accuracy of face recognition. Therefore, we propose a mask wearing recognition method based on MobileNetV2 and solve the problem that many of models cannot be applied to portable devices or mobile terminals. The results indicate that this method has 98.30% accuracy in identifying the masked face. Simultaneously, a higher accuracy is obtained compared to VGG16. This approach has proven to be working well for the practical needs. © 2022 ACM.